DOCOfiEHT BES0HE 

HE 007 597 

Berdahl, Bobert 

Criteria and Strategies for Program Discontintiance 

an^ Institutional Closure. 

Kellogg Foundation, Battle Creek, Hich. 

30 Jul 75 

43p.; Some pages may not reproduce clearly due to 
marginal legibility of original; Paper presented at 
the Kellogg In-Service Training Seminar (SH5E0 Snnual 
Conference Netr Orleans, Louisiana, July 1975) 

!!P-$0.83 HC-$2,06 Plus Postage 

Educational Accountaliility; Educational Finance; 
^Evaluation Criteria; ^Higher Education; Hodels; 
^Program Evaluation; *State Boards of Education; 
♦Statewide Planning; Tables (Data) 



Statetride review of nev academic programs in nov a 
fairly common phenomenon, but serious review of existing programs is 
a much rarer practice. Sov that economic conditions threaten to leave 
many programs stranded, ve face the uncomfortable fact that 
institutions are very sen^tive to questions raised about the 
legitimacy of existing programs. Even vith heavier institutional or 
ffiulticampus system involvement in the review of existing programs, 
there vill still remain several important roles for the statewide 
agency. They must not yet undertake self*evaluation; they must 
monitor the quality of the self-evaluation processes that are 
underway; and they must ensure that even high quality self*evaluation 
is placed in the broader context of state, regional, and national 
needs. Tventy-eight coordinating boards are engaged in some form of 
review of existing programs, trith ttro other boards having such 
procedures under development. Issues important to address are: (1) 
vhich programs to review; (2) the criteria for review of existing 
programs; (3) strategies and procedures for existing programs; (4) 
board start-up strategies; and (5) costs and benefits of review of 
existing programs. (Author/KE) 
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PROGRAM DISCONTINUANCE 

While statewide review of new academic programs is by now 
a fairly coimnon phenomenon^ serious review of existing programs 
is a much rarer practice. There are some obvious reasons for 
this. Until recently expansion was the name of the game^ and 
most board attentions were concentrated on which new programs 
were going where. If mistakes were sometimes made and new 
programs approved before there was actual justification for 
them, usually the incoming tide of expansion ultimately covered 
them up, IJow that the tide is turning and threatening to leave 
many programs stranded and visible as it recedes, we face the 
uncomfortable fact that it is doubly sensitive to raise ques- 
tions about the legitimacy of existing programs. Faculty have 
already been hired and some oJ them have become tenured; s^ 
dents have been acce^jted into the programs with expectations 
of 'finishing them, and some are occupying dormitory beds, 
helping to pay off construction bond debts; specialized library " 
or laboratory facilities may have been acquired. 

Above all, the notion of campus autonomy is much more 
deeply involved when it is a qnostion, not of denying yermis* 
siou for a new program, but of . recommending or ruling that an 
existing proj^ram be terminated. Academic programs lio at the 
heart of the enterprise and it should be undcx'staiuliible that 
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statewide movements in the direction of review of existing 
protjrams cause deeply felt concerns and even hostile reactions. 

Yet, given the need to iind the means for reallocation 
of resources, such statewide intervention seems inevitable, 
Tot one thing, the institutions themselves have not seemed 
anxious to grasp this nettle; Glenny's survey of institutional 
presidents (1975) indicated, for example, -fchat only 3 percent 
had engaged in extensive graduate program elimination or con- 
solidation betv/een 1968 and 1974. Another 27 percent reported 
that *'some'' elimination or consolidation of graduate programs 
had occurred. The corresponding estimates for the period 1974 
to 1980 were 7 percent and 50 percent, indicating a near doub- 
ling of intent. (Insert Table 1 here). 

The record seems somewhat better when one turns to system 
headquarters of multi-campus institutions. There Lee and Bovven 
(1975) report that since 1970 seven of the nine large systems 
they studied had initiated programs for the periodic review 
of existing graduate and professional programs. Five or six 
of these systems agreed that their review of existing programs 
had become more inclusive and more stringent, including considers 
tions of quality, costs and campus missions. Furthermore, seven 
of the nine systems estimated that their review procedures 
would become even more inclusive and more stringent over the 
next ten years. (Insert Table 2 here). 

However, even with heavier institutional or multi-campus 
system involvement with review of existing programs, there will 
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Source: L.A.Glenny, "Institutional Research in a Postsecondary V/orld" 

Associaltion of Institutional Eesearch Conference, St. louis, Ko* 
April 28, 1975 
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ACADLtllC PROGRAMS ; " ^ ' • , ' , ' ^ 

* Rcvifiw of Cxistinc) Prorjrams . . - ' ' 

1, Current Univcrsity\vidc procedures for periodic revlev/ of existing academic 
• programs 
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/ a. ' UndorgratJuatc programs 
\ ;.b. GracUmtc /professional 
' prograii^s 

2. "^State budgetary requirement of priority listing of existing programs 

a. , 7 Ho requirement , ^ 
^* l. <5 H^s been required, but not regular practice 

, ^ Is required and part of regular practice 
' 5*- ^ ^ Other ; ^ - ' 

3. Changes since 1969 in Universitywide review of existing programs 

. a. Little or no change ' ^ 

b. ^ More Inclusive nov/ 

c. ,C More stringent now, based on acadei^ic quality 
.. .r , tlore stringent now, based on carpus mission, etc. 

e. More stringent now, based on fiscal criteria 

/• - ^ Other 

4. Expected changes in Universit)'^/ide review 



a. Little or no change 

b. Moi-e inclusive/stringent 

c. Less inclusive/stringent 
D. Other 
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Source; E.C.Lee and P#M, Bov/en, T-Tanar:jr>r: rTulticn^nun Svfiten s, 
San Francisco; Jossey-13ass, forthcoming* 



still remain several important roles for the statowido agency, 
They must provide the only review for those institutions not 
yet undertaking self-evaluation; they must monitor the quality 
* of the self-evaluation processes which are underway; and they 
must ensure that even high quality self-evaluations are placed 
in the broader contexts of state, regional and national needs, 
Just as state interests sometimes clash with those of the 
institutions in terms of approving new programs, so there may 
occasionally be a disparity between institutional and state 
interests in the pattern of program termination. ' ^ ^ 

State Board Activities in Review of Existing Programs 

Subject to a more complete report next Fall when Bob Barak 
and I will have gotten deeper into our data, I would estimate ■ 
as of now that some eleven of 28 coordinating boards are engnging 
in some form of review of existing programs, with two other 
boards having such procedures under development. Of the 13 
consolidated governing boards, six seem to be underway in this 
area and two more report procedures under development. So, 
for each type of board, slightly under half are already in, 
or contemplating soon entering intc * this kind ol activity. 

The issue of actual board powers in this area is u slighLiy 
different question. in theory all consolidated governing boards 
would have such program review powers: what the board giveth, 
it can taketh away. Coordinating boards, on the other hand. 
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vary widely in their powers to review existing programs ^ ranging 
all the way from no mention at all of the issue in the enabling 
legislation, to granting power only to recommend termination, 
to authorizing final powers in certain fields or for certain 
types of institutions, to granting blanket regulatory pov;ers 
over all public higher education, to even folding in private 
sector programs as in New York. 

Some of the variations reflect the rich diversity that 
runs through this area (like so much of American higher educa- 
tion)^ making the Job of the observer attempting generalizations 
nearly impossible. For example, one state board with only the 
power to approve new programs nevertheless undertook a study 
and made recommendations concerning termination of existing 
ones and met with no institutional opposition. In several 
other cases, approval of new programs has been made contingent 
on their being evaluated after some time period like three to 
five years; the Doctor of Arts programs in Illinois were approved 
under these conditions, so the state board's limited reoommendiny 
powers were strengthened somewhat by this technique. Another 
pattern linking stronger powers over new programs to weaker 
ones over existing programs is cited by Lee and Bowen, but 
is not one that I could recommend: that is, to trade-off 
review-level approval of a desired new program for institutional 
termination of some existing one. That may have the '*ring of 
reality** to it, but I see it as no substitute whatsoever for 
satisfactory on-going procedures Ts^r reviewing existint; prot;rams- 
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There are also examples of boards lackint; general powers 

which nevertheless have limited powers over special subjects 

or certain institutional jurisdiet i ons , The Pennsylvania 

Board of Education can provide examples of both kinds: It 

has stronger review powers over the state colleges than over 

the other public institutions in the state; and it lias review 
new 

powers over /teacher education programs in all 86 institutions, 
public and private, which have such programs. 

Another interesting variation involves the role of tlie 
statewide board in cases of voluntary institutional termina- 
tion of programs. In Massachusetts the enabling legislation 
creating tlie statewide board could be interpreted to mean that 
this board would have to approve any institutional teniiination 
of prograiTis or degrees. And in Illinois, an issue has recently 
been raised as to whether, once an institution voluntarily 
terminated a program, state board approval would be necessary 
for its subsequent, re-introduct ion , (When the Board of Higher 
Education pressed foi the need for re^approval, an institutional 
spokesman warned that under such conditions, the university 
would never, never, never voluntarily terminate its programs.) 

The Illinois Board in 1971-72 was the scene of another 
variation in program termination, but the activities were only 
tenuously linked to the kind of revievv proce:r!fci I am conoern(*u 
with; in any case, they have been well analyzed and criticized 
in two articles in the Suinmer 1973 issue of the Education 
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Rocord (Flcutje and Sample; Lanier), Thus, I will just here 
note in passing that that board, faced with a proposed 
executive budget increase of only 5 percent for higher educa- 
tion, asked all public institutions to respond within a month 
with a list of their low priority programs, the sum of resources 
devoted to which would constitute at least 15 percent of that 
institution's operating budget from state Appropriated sources. 
Wot without some protests over both inadequate time and the 
method involved, most institutions complied and from these 
overall low priority lists, some $24. 1 mill Ions worth of pro- 
gram reductions were recommended. This constituted about 4.5 

percent of the base budget. Then this sxim was added to $36.6 

mill ion 

million of new money and $5*2/gained through deferral of capi- 
tal grants to constitute a recommendation for a new and improved 
program kitty of $65-9 million. The new priorities had already 
been established in each institution's proposed next year's 
budget. We might note that the termination of academic programs 
constituted only a modest part of the total savings and that 
the exercise has not been repeated since. (Insert Table 3 here). 

Which Progr a ms to Review? 

This is no trivial procedural question, as a failure to 
pay careful attention to this issue could result in the whole 
process becoming unworkable^ For it^ is not merely a matter of 
dealing one at a time with proposed new programs, but rather of 
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Table 3 



Summary of Prcgrammatic Recommendations in lOTZ'^yi Ofieiatins Budeet Retfuctt 



Program Increments 

Incfc^Mn;: Acccs*. , S26,3S4j81] 

Puhlt^ _tt IMC rsi tics 

Spcri.Ll Mtjipt>ft pffusrj^mt $ S.15,56S 

uMifcrKr:itJu:Mv enrollments 3 j 252 |S94 

Vn\tilt: oinutMir.ity crjlk-;:ci> 

K\;a(:t[t-4| vcrnllntcau : 6«76S,64{^ 

]>i*.iil\vt:it:i;fcd ^tmk-cita prognxm ^nxnts. , 1,400,000 
Exf^iduViJ vrhrthr^hip ont\ erant program 

for >iitt!i.Mts 000.000 

SuStf.Lii $ao,3s-Lijn 



Imprnvii:^ I'^tCMlty <iftd M<iip[oycc Dcciciits 16,425.915 

yii-rh ^i\ury Uun^i^i $12,22.1,0!S 

lnti.rtjvia hiiilih LciufiU 4,200 ,000 

SuliioiAl $16,423,913 

Hcilih l"M<ir,itif)n rrn^rants 

Ni;w,iiit! improx-cd j)ffi^r<ims $ 6,2(1,756 

Exp.iitilnJ t-iu-iilhr.cniA ^2,1 tO, 200 

t Su\-UiU\ ■ $ 0^/"^ii!56 



8,393,956 



New Sctii^iT ItwiiMiior^s 

:kt\A v\\y.iiuh-t\ :\c:%t\^tn\c pro^romt. 
Dcvr!fp*Jifiil;il pro^fnnis. 



$ 3,r/)5,32] 
1.172,714 

Suliwiul $ 4.£I6S,035 

Improvif^;* ri:i1rn:r<idtintc [iducation 

Linivrf^ltuit 

$ 



I'llMir iinivrf^ltuit 

Ni-w atiii utipr^/vcil ;*<rf):;r.itns 

Itttinv tM^v i^fi(;r<'ttti njjtu^ns ^ 

Vtiltitii.try jih>>ir<^l 1-4 1 tic .i lion pro^r^imtt < 

Vtjr.ti[ii;L:il-tcch:)ical proj^niin incentives.-. 
Sm1*I(i|31 « . < 



4,863,035 



4,606,553 



0.SSJI6 
412, a72 
ass t 535 



1,2 50.000 
$ 4,66~6,.';53 



Icnptcmi-t.; iit,* Master rj;iii— Ph^ivc lU 

Ncwpr<,^ri((n> $ 010. ."192 

InicritMHuiJonaJ pro^^r.ini grants 1 ,000,000 

Conr)mu:.ity bcrvtcc program grants 70 0,000 

Subtotal Vii6I0,592 

Qpcratir; and Maintaining^ Sew Buildings 

Toul 



2,610^392 



2>68I>DI2 
$65,938,674 



l*roj:ram Decrement! 

Rvdiietn^; Overhead ExpendUurcii ^ $ 9,397*431 

Eliminating; or Retluein;? DupUeate or LowNeed Pfoj:rams. , 5,1 13,5t4 

C^radtiatv prMir.im* $ 2,314,000 

Tc<^dii-r cdtieation profrranis...: Ij 745^000 

Other specific dppUcate or )ow-necd 

projjrain^ \ ,053,644 

5s;)btoial S 5,113,5:4 

EllminatiJi^r Involuntary Phy^iical Eiducation 2r362|$S9 

Elimicaiini; Programs Out:i{dc In^ttj tut tonal Scope and 

Missba 419,756 

EKmir.atini; Underutilized Computer Capacities 3,539,000 

Moving Ujiivi-r^tty Hospital Toward Uscr-Cc^^t Oasts 1,700,000 

Net Rccl^jciinn for Nonrccurrjns^ expenditures, for Proj^rams - 
No Longer ncqutrlug Funda. and for Certain 

Z*^-ro-ba*cd Items |.5S3.0?D 

Total $24,116.6:0 



Source: H.l^.Flentjc and S.Q. Saaiplet 

''btatGwdie deallocation Tlirough 
Proiiran Priorities*'' Edtication&l " 
. . Kecard ,. Sunncer 1973.* 54,(3),. 175-184* 
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reaching into the morass of existing programs, offered in 
nearly infinite variety, with parallel curricula oftoh wearint? 
different labols, and selecting according to some apparently 
Justifiable rationale those few programs which can be evaluated 
in depth at any one period of time. An attempt to approach 
review of existing programs on a more massive scale would 
probably result in either great superficiality or an overload 
of board capacity and/or budget, (We will have more to say on 
the costs of review of existing programs in our later report.). 

No surprise, then, that except for a University of Wiscon- 
sin system review of undergraduate programs currently underway, 
and completed reviews in both New York and Wisconsin of, masters', 
degree programs, all state reviews (including others in I^ew 
York and Wisconsin) pertain to the level of doctoral programs. 
The universe there is the most limited; costs are higher; 
unemployment issues are more drg.matic; and quality considerations 
seem more amenable- On the other hand, Job market aspects often 
transcend state boundaries, particularly in smaller states. ^ 

Of course, even within the restricted doctoral field, 
further selection is necessary. Most states engaged in this 
process , seem to rely on quantitative indicatnry as tripwires 
to bring certain programs before the institutions and thv state- 
wide board for in-depth evaluation. After some early cxpericncoG 
with what one questionnaire respondent called *'inst itut ional 
suspicion and paranoia , " some states stretiscd that programs; 
thus selected for detailed evaluation were under no presumption 
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of being "in trouble"; that to the initial quantitative criteria 
would be added qualitative and other dimonscions; and that a 
whole host of possible extenuating factors would be considcrod 
before program termination decisions would be reached. Quanti- 
tatlve indicators often included program productivity, costs, 
student demand, and extent of program duplication within the, 
state. ^ 

West Virginia uses a mixed set of indicators and a very 
particular way of dealing. with them. Using cost, output, insti- 
tutional priority and program quality , each institution is askod 
to make forced ratings about each program. Sixty percent of^ 
the programs should be rated as normal on each of the four 
variables; twenty percent should then be placed on either side 
of the normal range. Each institution should then select which 
programs it wishes to have reviewed in depth. 

While the above methods offer various ways to get an 

administrative handle on a difficult problem, they suffer from 

the common defect that the program evaluations tend to be oi 

programs 

isolated doctorate/ at different institutions and do not neces- 
sarily involve the statewide orientation and competences of 
the state board. Ideally, at the very least, all doctorates 
in a given subject area would be evaluated at the same time; 
at the very best, this would be done for whole fi^=>ld clusters 
at a time, in recognition of the iu Lerdependciice of advuucod 
work in related Ciolds. The New York state review starLed 
out with pilot studies in single subject areas: u history 
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review for the humanities/social sciences, and a chemistry 
review for the natural sciences. With later reviews in as- 
tronomy,. English, and physics, criticisms were made of the 
single subject approach and next year's review will be of all 
foreign language doctorates and those in comparative literature 
The Florida Board of Regents has some proposed new guidelines 
which anticipate using HEGIS clusters as the means of selection 

One final observation of program selection: a Wisconsin 
document speaks well to the danger of allowing the quantitative 
trip\yires to be the exclusive channel through which programs 
are chosen for more intensive Analysis, 



For a v!:^"±ety of Judgment'arl reasions, administ r*^ tot*£5 
or planning committees for a particular institution 
might ask for more intensive review of a program which 
had not been selected through the audit process. For 
example, the anticipated retirement or departure of 
key faculty members in a given program might create 
a presumption that review* of the program should be 
undertaken prior to restaffing; or a regularly scheduled 
site visit by an accreditation team might catalyze an 
institutional audit and review of a program in conjunc-* 
tion with the preparation of documents for the visitors; 
or planning studies concerning the minimum staffing 
which should be maintained for the essential programs 
of an institution might generate need for review of 
particular programs; or recommendations from system- 
wide or institutional task forces on curriculum changes 
could also generate need for program review. (quoted 
in Barak, 1975), 



Criteria for Review of Existlnp; Programs 



It would be poss^ible to make this into a very lengthy 
paper ringing the changes on all the combinations of criteria 
being used or proposed by state agencies. But to some extent 



that has already been done through numerous extracts from 
state documents included iij Bob Barak*s paper, '*A Survey 
of State-Level Academic Program Review Policies and Proce- 
dures for Higher Education/* April 1, 1975; Furthermore, 
our* planned monograph will be a much more appropriate vehicle 
for the nuanced details which may be of considerable interest 
to others around the country engaged in roughly the same 
pursuits (at the multi-campus and institutional levels as well) 
Here, then, I propose merely to do two things: 

1) list the ten criteria for program discontinuance 
suggested by the ECS report, Coordination or Chaos , 
(1973), along with a tally- from Barak-^s paper on how' 
many states responding to his survey were using each 
criterion; and 

2) reproduce two pages from the Final Report of the 
Montana Commission on Postsecondary Education (1974) 
which lists some 22 possible criteria to be employed 
in reviewing existing programs, plus an additional 
eight when graduate programs are involved! If most 
of these criteria were to be^ applied seriously, the 
Montana Board of Regents could probab^ly use enough 

> additional staff to lower the unemployment rate by 
several degrees . 
The ten criteria suggested by the ECS report and the 
number of states reporting to Barak to be using them are as 
follows ; 
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1) the number of graduates from the program in each of 
the last five years (14 states); 

2) the number of students enrolled in the program (entry 
and dropout rates) (10 states); 

- 3) the size of classes and the cost of courses identi- 
fied as integral elements in the program (4 states); 
. 4) cost per program graduate (B states); 

5) faculty work load (2 states); 

6) program quality as reflected by its regional or national 
reputation, faculty qualifi^tiohs' and the level of ^ ' 
position achieved by gradCii^'tes of the program -'(8 
states) ; 

7) total production of a program' s graduates from all 
institutions in the state, region and/or nation 

(2 states) ; 

8) the economies and improvemeiits in quality to be 
achieved by consolidation' ahtf/or elimination of the 
program (5 states); ^ - 

9) general student interest and demand trends for the 
program (9 states); and 

10) the appropriateness of the program to a changed insti- 
tutional role or mission (9 states). 

The thirty possible criteria listed by the Montana Commission 
are as follows: 



Appropriate criteria for the review ot existing /voffrnms will Iw 
dovoloficd ovOr a pcrioU of time and Kvif/hc sa/>/act to cliange 
cryndtVons niter. Tiieraforc. wo iiesttntc to spociiy tiiOtn ttut t>ctieve 
itiey shouid take account of ttie lottowing factors: 

0. number of gradaatos from the program in eacti of tiie last 
five yosrs. 

b. number of students onroiied in tt^ program for each of 
the last five years, rate of conjpietto$h tiie rate of 
attrition^ ratio of enroffment to degree, productivity, 

C. the number of students not enroUed in the progtam but 
wfio were served by it for eaci} ol tiia^iast five years: 

if. the size of classes identified as integral elements inthe 
program. ^ 

e* for colleges, universities and community colleges, cost 
per credit hour ol the courses identified as integmt 
efements in the program (upper division, dower division 
and ffraduate}. 

/ for vocational-technical centers, Qost per contact hours 
for- courses identified as integral elements in the' 
program. *w ' 

g* cost per program graduate. . : ' . 

A. faculty/instructor v^rkload- * " 

I faculty/instructor qualifications. 

/. reputation and intrinsic velue of the program. 

it. positions achieved by graduates of the program.-* 

£ positions attained by persons enrolled in the program 
wfjo may have achieved their educational ob/actives * 
withotn completing requirements fcff^tfie- degree or 
certificate. 

m, total production of graduates in the program area from 
afl institutions in the state (and when appropriate, in the 
region and/or nation). 

ru economic and/or qualitative improvements which might 
be achieved by consolidation and/or elimination of the 
program. 

* o. general student interest, evalaation^nd-demand for the 
program^ morale of students in the prfigrjarru 

p. indicators ol present andluture demand fbr graduates of 
the program. - :/t 

■ igr.' appropriateness of the program to (he-mission of the* 
institution. ' '.^ y 

fl any needs for other programs of higher priority which 
' ' mightbe funded fully or paaially Irom savings realized by 
' discontinuance ol the program under review. 



. i. 
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^deijiJacY of support services, particafarfy fibrory, 
(aboratory pnd educ^thiwl tQCiiiU'es, 

t compatibiiity with state plant. T ^ ■ . . 

t II. similarity to programs ollored at any af-tbe other units. 

V. relevance ol the program to Its obfective. 

> in addition, the following criteria ^houfdbe applied to the review of 
graduate programs by the fiegents: 

a, averagetime of completion of those to whom the degree 
has been awarded, 

Ik benefits accruing to the institution and the state 
independent of enrollment or degree production. 

c. proportion ol departinantal^resoOrces devoted to the 
program. 

sources ol landing — state, federal, etc. 

> a qualifications of faculty. ^ . 

/- qaalilications and backgrounds of students attracted to 

> the program. ] 

g. relatiot.jhip to tho impact upon undergraduate program. 

h, ai/ailabtlity of similar graduate programs at other units. 



(Montana Con:i!aissLon on Pbstsec^^;*civv Ktiucaticni^ 197Vl, pj>.37-3;0 
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The National Board on Graduate Education, in a passage 
quoted with approval by a recent ECS Task Force on Graduate 
Education (1975), commented on dangers associated with over*- 
simplified use of quality, productivity and manpower criteria: 

1^ A single measure of quality should not be applied 
to very diverse programs — programs that may be serving 
the needs of nontraditional students for nontraditional^ 
forms of graduate education, t^ultiplo indicators of 
qrality, sensibly related to different program missions, 
should be developed. 

2. Statewide planners should resist the temptatlcJn 

to apply simplistic formulas to doctoral programs, such 
as "eliminate any program" -that haa not produced more * ^ 
than two doctorates withiia the last two years." Such 
statistical measures may flag programs in need of- re- 
view, but no program should be eliminated on the basis 
of simple statistics alone. 

3. When evaluating graduate prograjoos , planners should 
not attempt state-by-state labor market analyses, since 
the mobility of the highly educated is certain to con- 
found such analyses. A more appropriate criterion, we 
believe, is assured access to graduate education for 
residents within the state- (or -within the region, 
through reciprocal programs). 

Just as we urged the federal government not to over- 
react to current labor market imbalances, so we also urgn 
state governments to take- a long— run view in supporting, 
graduate universities. The lengthy process of building 
excellent graduate programs can be undone very rapidly ^ 
and when these programs need to be built again, as some 
of them surely will, the costs will be enormous. (see 
National Board on Graduate Education, 1973, p. 20). ■ 

* 

Such disclaimers on manpower planning may be well meanings 
but in some fields like history doctorates, the employment 
picture has become so grim that one feels such considerations 
must be playing at least an implicit role when it comes, for 
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exajnplo^ to Judt;ing what to recommend for history doctoral 
programs of marginal quality. But implicit criteria make it 
difficult for the institutions to read all the signals. 

Strategies and Procedures for Review of Existing Programs 

* 

State practices vary so markedly " here that generalization^ 
are difficult. So much obviously depends on the particular 
circximstances of each state. Thus I shall merely offer obser- - 
vations on the various groups ^ involved in the process and^ 
then try to examine some related emerging issues. ■ 

The state board staff obviously have a major role to play, 
and no agency should venture into these troubled waters^ unless 
it is well staffed in this area. This means persons of matui'ity, 
sensitivity and Judgment, preferably with some years of experience 
in the academic side of universities or colleges. Once this 
policy area has been opened up^ it may -be hard to let go of it with 
out having invested tremendous amounts of staff time. And the 
more senior the staff involved, the better.- A major reason 
why the controversial New York review of doctoral programs has 
not been more explosive has been the intimate involvement from 
the beginning of Ted Hollander, Deputy Commissioner for Higher 
Education. Even the strongest critics of the doctoral review 
concede that he brought to his role in this t:isk a reput:\tion 
as a fair and exceedingly competent staff member. 
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Tlie staff rolo can be larf^or in small states where the 
program review problems may be of a more manageable scale or in stattM 
where it may have been possible to build up mutual confidence 
and trust between the institutions and the central board. The 
staff role will also tend to be larger in states which choose 
to rely primarily on quantitative criteria for evaluation. In ^ - 
such cases they will need to rely heavily on the accuracy of 
the state*s information system, and they must learn to treat 
this information with considerable sensitivity. Premature 
leaks of evaluations concerning terminated or probationary ^ 
status. can prove politically very explosive. We even heard 
some suggestions that the ominous term ^'probation" be abandoned 
because of the damage to the morale of the faculty^ students 
and institutional administrators involved with the program in 
question; some euphemistic label such as ^'acceptable — under 
development** was preferred. The board staff relying primarily 
on quantitative measures dealing with program productivity will 
also have to be alert to the danger that, in the absence of 
quality controls, they may be encouraging endangered programs 
of low productivity to lower standards in order to generate a 
higher body count. 

Finally, I would urge all parties to the process, but 
particularly the board staff, to keep very much in mind the 
desirability of trying to find positive incentives for insti- 
tutional cooperation. This can bo done more readily insido 
the Jurisdiction of a consolidated governing board where access 
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to a whole host of system incentives (such as funds, flexibility^ 
etc*) lies alongside the program review process. For coordina- 
ting boards with only advisory powers and no major budget 
leverage, it may be more difficult, but I would still urge 
this as a major staff concern. 

The use of outside consul^tants becomes more commori when 
the evaluation criteria are qualitative. It seems too explosive 
for the staff or in-state consultants to hazard these judgments. 
But outside consultants are not. without theiT problems "too. 
For one thing they normally lack an ^intimate" knowlexlge of inter-" 
institutional relationships or the general pattern of the state- 
wide context. Yet isolatied\judgments of program (Quality really 
need to be linked to broader policy considerations. This 
raises questions about thre relations between outride consul- 
tants and both the staff which provides orierftation for their 
efforts and any statewide, institutional advisory, committee wb^ich 
will be reviewing their report. During the first pilot runs 
in the New Yorlc state doctoral program review, there were a 
few comments to the effect that both the State Education Depart- 
ment staff and the Doctoral Council reviewing the rating 
committee recommendations were perhaps too much in awe of th^ 
academic "stars" who hfid been used as consultants. As a con- 
sequence, the history rating team was allowed to disregard 
guidelines urging attention to teaching excellence as well as 
other factors; it focused e^cclusively oa traditional measures 
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of quality: two books published per faculty member, faculty 

depth in most major fields, appropriate library facilitiOK 
and full-time graduate students. This resulted in a recommen- 
dation to terminate the program at St, Johns University which 
claimed that it was serving a part-time urban population as 
a definite service to the statewide goal of broadened access. 

There were also statewide ramifications to the initial 
history rating team recommendations for termination at 
SUNY-Albany and probation for Syracuse University and SUNY- * 
Binghampton. Although the Binghampton finding later under -appeal 
was altered to approved status, some critics said that the 
original verdicts showed little concern, for. matters of geographic 
access to doctoral programs for students between the New York 
City area and Western New Jork. 

Another example of the nefed to keep the high-powered con- 
sultants' enthusiasms under control was in another reversed 

early finding to approve most parts of the SUNY-Buffalo history 

its 

program, but to recoinmend phasing out of /doctoral work in 
Latin-American and Asian areas. Such selective fine-tuning was 
considered excessive state intervention and subsequent rating 
team findings were confined to recommondations for departments 
as a u'hole. 

One should, on tlio other hand, note that there was wide- 
spread praise for the quality of the consultants used in Now 
York, that tho State Education Department formed the r.;-GO 




member consultant panel for each subject with help from tho 
institut ions / and that each chairman of a department about 
to be reviewed was allowed to veto from the panel the names of 
* persons considered undesirable as site visitors for any 
reason I. 

New Jersey decided to use-.putside consultants in its 
review of masters programs in the state colleges. Listed 
below are the criteria to be employed in selecting the 
consultants: 

*1. recognized authorities in theix fields 

2. knowledgeable about aiid concerned with education ' 
in their fields 

3. likely to understand -readily the role and problems 
of state colleges 

4. likely to help faculty and others develop construe- 
tive suggestions for improving programs 

5. likely to appreciate the merits of differing opin- 
ions and approaches in a field 

♦6. not related to the institution offering the program 
for evaluation and not associated with any educa- 
tional institution in New Jersey 

, 7. not primarily associated with an institution from 
which a significant percentage of the program's 
faculty received their professional training 

8. free of any other pei*sonal, institutional or ideo- 

logical obligation that migjit be supposed in the . " 
normal case to affect impartiality of judgment. 



♦Items marked with an asterisk are taken verbatim from "Procedures 
for Evaluation of State Collej^G GraduaLo Programs*', Item 2, 
paragraph b. These are criteria of eligibility, and presumably 
no exceptions would bo made except in the clearest case, with 
general consent^ when no suitable alternative is available. The 
other criteria raise more difficult quosLions of interpretation, 
but might be considered important criteria of desirability, 
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Although I can see its becoming a sensitive kind of opora- 
tion, SUEEO mif;ht consider keeping a central file on state 
experiences with outside consultants, with recommendations, 
* confidential or otherwise, based on the extent to which the . 
subject-oriented experts also possessed some sensitivity to 
institutional and statewide concerns. As more and more insti-- 
tutions and multi-campus systems join state agencies in the 
business of evaluating quality, there will probably be a 
shortage of outstanding disciplinary scholars who are also 
sensitive to the broader dimensions-- The use. of consultants 
will, of course, add to the bill for evaluation, but up to a 
certain* point at least, it seems worth it. (The New York 
doctoral evaluation budget was $59,000 last, year.) 

Nor is there any respite from complexity when we come to 
consider the use of an inter-inst itut ional advisory committee 
in the review process. Two states responding to our program 
review questionnaire reported that they had started out using 
only staff analysis but soon found that institutions cast as 
the passive victims of the review would appeal negative reccim- 
mendations on both procedural and substantive grounds, and 
sometimes win board reversal of staff findings. In an area 
as sensitive as I have already tried to depict, this to be, it 
is absolutt?ly essential, no matter how great the state presyuros 
for early and/or drastic results, to lean over backwards to 
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grant tho institutions full procedural duo process, and to 
get their maximum contributions to the substantive wisdom of 
the results. 

Admittedly, some of the institutional "suspicion and 
paranoia" (as earlier quoted from one respondent) is more "for 
the record" than a genuine position. Some- institutional presi- 
dents have privately admitted that because of their relations 
with faculty, students, trustees, and alumni, they were forced 
to take public positions critical of a program discontinuation 
process, whereas in fact tfaey very much welcomed gfetting some - 
external leverage to help them crack open entrenched internal 
positions. Yet, clearly, the larger part of institutional 
concern is real and merits a serious response. 

It is also important to realize that responding to such 
concerns by creating an inter-institutional advisory committee 
is only the first step toward meeting the problem. In an 
earlier book on statewide coordination (1971, Chapter 7) » I 
discussed at some length the various dangers with runaway 
committees, log rolling, Have-Nots ganging up on the Haves, 
etc. Suffice it to say here that, particularly in the early 
months if it happens to be a new committee, there will have to 
be a disproportionate investment of senior staff time if this 
crucial body is to be groomed to play its proper role. On the 
one hand, the staff must see that the centrifugal forces do not 
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como to dominate its dolibprations ; on tho other hand, tho staff 
miifjt be hypersensitive to the danger that either in reality 
or in appearance, the advisory committee comos to bo viewed 
more as the "lap dog" of the central staff than as a major 
source of candid criticism from the institutions. Realizing 
that there will be dangers either way, I would urge serious 
consideration for the possibility that some^^ institutional repre- 
sentative and not a member of the central staff chair the 
meetings . 

One of the interesting aspects of the New York doctoral 
review process has been the role of the Doctoral Council, a 
13 member group of graduate deans (or equivalents), with two 
from SUNY, two from CtJNY and nine from private universities. 
Its predecessor body endorsed the need for the prestigious 
Fleming Committee to malce its report "Meeting the Needs of 
Doctoral Education'* (1973). This report provided the basis 
for the Regents Position Paper No. 19 which in turn led to 
the present doctoral review. The institut ional representatives 
on the Doctoral Council have, then, from the beginning been in 
on the process and most of them continue to support it today. 
Some have had some difficulties with their home institutions 
Avhich seemed to fear that they had been too "socialized'^ to the 
statewide point of view. 

The Doctoral Council, chaired by Ted Hollander, played a 
major role in reviewing the early pilot runs, offerinf^ sub- 
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stantive as well as procedural recommendations, A few insti- 
tutions, however^ were concerned that if this body, with 
their representatives on boards took, substantive positions, 
they (the institutions) might be inhibited in later expressintj stron,. 
positions to the contrary. So there was a period during sub- 
sequent subject reviews when the role of the Doctoral Council 
was reduced to that of commenting on matters of due process 
only. This, though, created a situation in which a marginal 
strengtheaiQg of an institution's freedom to" react after the 
Commissioaer's decision on the rating committee ' s recommenda=- 
tion (now tempered only by staff aduice) had been purchased at 
the price of the iastitut-lons ' collective advice to the Commis-^ 
sionar before his decision. According to reports Just received 
iQ July, the Doctoral Council will revert back to its former 
status of rendering both substantive and procedural comments. 
If all this sounds complicated, that's because^ it is! (The 
New York review process is further described in the Chronicle 
of Higher Education of January 27, 1975 and in an excellent 
article by Amy Plumer (1975).) 

Concerning any roles for members of the statewide boards, 
I would guess that, sooner or later, given the controversial 
nature of the process, they will be involved in one way or 
aaother. One respondent to our questionnaire reported that 
his board members felt distinctly uneasy in the face of the^o 
intimate academic matters. On their consolidated governing 
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board, thoy had been used to worrying about budgets and build-- 
ings and football teams. One answer to this problem, not 
without dangers of its own, is to create a standing committee 
of board members on academic programs, so that over a period 
of time and with careful staff assistance, some members can 
build up a real sophistication in these areas. Hopefully, 
they will not become so sophisticated that v.t hey will be tempted 
casually to substitute their own private preferences for care- 
ful judgments by outside consultants and board staff! 

Board Start-Up Strategies 

In terms of general strategy questions, boards contempla- 
ting starting up the process might ask themselves whether it 
is better to commence in low key or to come on very strong. 
Some shock-ridden New York institutions might plead fof the 
former, on a variety of grounds. For one thing, some- were 
surprised to find that so-called pilot runs were leading to 
immediate operational decisions. The pilot studies, cim pilots 
accomplished their purposes, for subsequent subject reviews 
were much improved processes. But high degrees of anxiety and 
some hostility might have been greatly lessened during the 
admittedly imperfect early procedures if all concerned had 
not seen real bullets placed in the chambers of Commissioner 
Nyquist's weapons. 
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Another argument for a mellow beginning to the review 
process is that negative findings constitute, in a certain 
sense, evidence of some boards' own earlier acts of negligence. 
The New York Regents, for example, had formal roles of approval 
for most of these doctoral programs on the way up; if they, 
along with most of the rest of us, can be excused- for not 
having anticipated problems of excess supply, there is no 
such general excuse for their having earlier taken their role 
as **quality-controllers" so casually. Having mld-wifed, as 
it were, at the birth of some of these marginal doctoral pro-^ 
grams, they might be considered to have a moral obligation to 
exercise their relatively recent conversion to the cause of ^ 
very high standards with some compassion and circumspection. 

Some institutions have also opposed immediate operational , 
decisions on subject fields in an ad hoc manner* Rather they 
urge, as mentioned earlier, that related fields be evaluated., 
in clusters; that recommendations on given subject fields be 
given to the institutions or multi-campus systems in question 
with the opportunity to provide a coherent institutional response, 
taking Into consideration the relation of the field In question 
to the central core missions of the campus, its relations with 
other Inter-dependent fields on campus, and, if possible, fol- 
ding in the particular response to the institution's (and 
system's) own long-range planning efforts. While such a pat- 
term might 5 nvite lengthy institutional minuets before '*Act ion 
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with a capital "A" finally occurs^ it would f;o a lonj; way 
toward restoring the concept of institutional integrity in 
the prot^ram review process. And there would be nothing at 
all to prevent state boards with adequate powers from still 
finding at the end that the institutional response did not 
satisfy all the necessary concerns- What would have been lost 
would be time and some possible savings from earlier rather 
than later termination. But within reasonable limits issues 
of time should not be used to bypass - important procedural 
points unless elements of outright fraud require immediate 
intervention. And I happen to believe that the net savings 
to the state of earlier program termination are quite modest. 
(The SUNY-Albany history graduate program, for example, has 
about 35 students, about 2 to 3 of whom receive doctorates 
eaph year. There had been plans- to increase the output to 
5 or 6 per year. One doesn't know what staff reductions might 
occur among the 22-25 faculty if the doctorate is eliminated. 
One suspects there would be few dismi#5#5als but rather a free:?;p 
on new hiring and some shifts in teaching assit^nments. ) 

On the other hand, consumer prot6ctaon and fiscal retrench- 
ment are popular items these ddys in state government circles; 
and a board struggling for political survival may feel in need 
of surrendering some institutional goodwill in order to purchase 
some life insurance. There is also the academically much more 
respectable justification that a board starting out with high 



31 



30 



standa,rds rigorously applied is much more likely to oncourago 
institutional and mult i-campus system reviews that transcend 
cosmetic pleas for improvement of quality and result in actual 
program retrenchments. There are already some signs of this" 
in Hew York, where, facing not only the push of Regential doc- 
toral reviews but also th?e pull of state and New York City 
budget cuts, SUNY-Binghampton and StJNY-Albany and the City 
University of New York have all engaged in serious procedure^ 
for internal retrenchments ' (For example, 40 percent of CUNY's 
masters* programs are beiirg cut.) 

Finally, to return to political considerations, Machiavelli, 

who too often has been proven uncomfortably accurate in his 

assessment of political gains and losses, has advised that 
when unpleasant tasks remain to be done, they should be under- 
taken aggressively at the outset* before the recipients* ability 
to resist has come to focus. Then, subsequently , when the 
situation permits an easing of the degree of intervention, 
many of the other parties will respond to the relative improve- 
ment of affairs (e.g-, the movement of the foot from the neck 
to the stomach), and life can go on w th a medium degree of ' 
intervention and the recipients' grat^ wde for the lessening- 
of tensions. It is hard to talk thus ith my tongue in my 
cheeky but lest anyone mii^iundcrRtand me, I personally favor 
paying the price of the mellowci' approach to program review- 
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Costs and Bonofits of RevlGW of Existing Programs 



The paucity of the following remarks will reveal: a) that 
I am no economist; b) that not much has been written on the 
subject by others; and c) that the whole area may be too new 
to permit meaningful assessments. It seems clear that the 
major costs will be th$ heavy investment of staff time, 
administrative support costs for the inter-institutional 
advisory committee^ aad the use of outside consultants^ if 
any. (The New York budget for this item, already mentioned, * 
was $59,000 per year.) NO board with a very. tight budget or - 
overworked staff should undertake this as a new assignment un- 
less it is able to let go something else substantial, (Retrenche 
retrench thyself!). 

We are also pretty hazy as to the exact benefits of the 
program discoatinuation process. One respondent stated quite 
pessimistically that very little savings resulted, as most 
faculty were merely transferred to related degree programs, 
(The SUNY-Albany faculty and student numbers were given above.) 
Oq the other hand/ Max Bickford reported from Kansas that their 
process had resulted in the elimination of 81 programs. Surely 
that has allowed some significant savings? There are also 
possible benefits already alluded to above; namely^ that in- 
stitutions and multi-campus systems will follow the lead of 
State board procedures and engage in serious program reassess- 
ments in many fields other th^n those chosen lor statewide 
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evaluation. Finally^ there may be some not illegitimate 
public relations pay-offs, as the state's academic reputation 
grows and as state officials may find their confidence in 
* the system increased enough to be willing to fund new programs. 

Emerging Related Issues ^ 

Let me comment briefly on several related items. First, 
I would hope that careful attention will be paid to the inter- 
face between program termination and collective bargaining. 
For the moment I have not heard stories of actual interaction,' 
but when program terminations begin to be more common and re- 
sult in some faculty dismissals or reassignments, ^you can be 
sure that negotiating agendas will not be far behind. There 
is also an Interesting side issue here; if, in fact, a stat© 
board has allowed an institution or multi-campus system receiving 
a program termination recommendation to reply that all necessary 
internal resources will be mobilized to Impvove the program 
in question, how is. this radical upgrading to occur? Often 
an entrenched block of mediocre faculty is the heart of the 
problem; yet any attempt to bypass that power structure may 
result in violation of a collective bargaining agreement 
protecting the faculty's right to self-government within the 
department, I leave that issue for organization theorists 
and lawyers to wrestle with. 
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Another area that will probably receive more careful 

attention in coming year& is that of exploiting the poten- 

the solution of 
tial oX eonsortia and interstate agreements to/the problems 

• of program retrenchments. Up to the present, to the best of 
my -knowledge, consortia and organizations like SRI£B, WTQIE 
and NEBIIE have been more useful at handling "add-on" programs 
eooperatively than they have with respect -^o existing pro- 
grams being phased out. But there is eertalnly a potential 
there worth investigating, Votruba^- ( 1975) has written a 
report on regional organisatidns for thd ECS Task Force on - 
Graduate Education. 

While New York state may be unique in giving its Board 
of Regents such sweeping powers over" the private sector, I can 
see the possibility of some private sector issues emerging in 
other states with respect to program termination. We know that 
approximately 44 states now have programs giving state aid 
directly or Indirectly to private institutions within their 
borders, 'S% also know that as some private institutions in 
deep financial trouble stagger close to the edge, they and 
their supporters ask the state for emergency bailing-out. 
What, one may ask, will the institutions in the public sector 
be thinking as some of their own programs are recommended for 
termination while related programs of perhaps similar quality 
at private institutions receiving substantial state subsidies 
are exempt? Methinks that such private institutions hud best 
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not assume that state monies can be acquired in any significant 
amounts w^ithout surrendering sizable amounts of institutional 
sovereignty. Whether or not this will go so far as to include 
state recommendations for program terminations in the private 
sector^ I do not know; but certainly one can anticipate demands 
for it from public sector institutions. 

Finally, I want to raise questions about the relationship 
between state board review of existing programs and the marked 
increase of legislative pxogram evaluation. Two good articles 
in the Spring 1975 issue of Higher Education lii New Bng4and v 
(Glenny; and Pingree, Murphy and Weatherspoon ) provide the 
context very nicely. According to these authors, legislators 
are becoming dissatisfied with merely voting the budget or 
passing a law and not later finding out what the actual pro- 
gram impacts were. Distrusting the executive branch, they 
have now in some 28 states set up post-audit oversight activity 
They wish to use tUe performance infonnation thereby gained ^ 
to help them make reallocation decisions in the present con- - 
text of fiscal austerity. Thus far, report Pingree, et,al,, 
during the first five years of formal operation the impact on 
higher education has' been limited; but the handwriting on the 
wall, they say, is clear; 

If administrators don't begin now to provide relevant 
and useful program data, state legislatures may well 
attempt to impose various sanctions, such as elimina- 
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ting or drastically reducing programs which do not 
exhibit positive olXocts. The lOKislaturos may fail 
In those efforts, but tho result will not be a public 
relations bonan:ia for higher education. 



...public higher education miglvt best bo served by 
accepting, however reluctantly, the legitimacy of 
, legislative program evaluation arid by meeting appro- 
priate criticism with speedy remedial action* 

night I suggest a different ending to thd second paragraph? 

Public higher education might best be served by ".♦.state 

boards demonstrating to legislators Und'^go^^ernors that -they * ' 

can take enlightened leadership in belpiag the higher education- 
I. 

conunutiity undertake careful re^llocatioti of resaurces to 'meet 
changing societal needs." Syrely review of existing programs 
would constitute a major element in the over-all process of ^ 
reallocation. Let us hope that the institutions, whose coopera- 
tion is so vital to the success of program review, -will see 
that genuine cooperation with the state board just might pro«^ 
duce results which would be sufficiently impressive to keep ' 
detailed legislative program review arm's length. Of course, 
the chances are real that even such efforts may not suffice, 
In which case we should all pray for^ high q"uality staff in the . 
legislative audit divisions.^ fiut I have Mong felt that there 
is a better chance for a state: board bluffer rdle iri the program 
review area than in that of state budgeting where political 
pressures seom overwhelming. If theso observations on state 
board review of existing programf5 will help improve the odds, 
then It will have been well worth the effort. 
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INSTITUTIOflAL CLOSURE 

K Institutional closure is not merely program discontinuation 

"writ large;" it is a much more politicized issue. 
II.' Responses to our questionnaires indicated that in most states 
there has not yet been an effort to develop state policy 
guidelines anticipating this problem. 
Ill* But in a few states the issue has emerged formally. 
A. Public sector: ^ 



1. 


South Dakota - Dick Gibbs'- - 


2. 


Montana - Pat Callan and Larry Pettit 


3. 


Wisconsin - Governor Lucey and John Weaver 


4. 


New York - Pusey Committee{public and 




private sectors) 


5. 


Oklahoma - viability guidelines - E.T. Dunlap 


6. 


Michigan - forecast of financial requirements 




to 1980's - Bob Huxol 


Private 


sector: . . _ . 



1. More privates closing but Ostar says also many 

new ones opening 

2. Grin) projections - e.g. New Y&rk- --Hollander's 

"Curiouser and Curiouser" - Cazenovia, 
Eisenhower, Finch, PIB College of White 
Plains - Pusey Committee 



1^ 
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3. AlabaiHa - Athens College - John Porter 

4. Florida - New College - Bob Mautz 

5. Ohio - Rio Grande merger - Dolph Norton 

6. Broader context of state pblidy vis-a-vis 

private sector. 

IV. Role of Statewide Board in a Highly Politicized Issue. 

A. Develop general guidelines in advance 
B* Qufckly mobilize relevant facts to any particular 
cases 

C. Make policy recommendatidn (oirly -if requested?) 

D. Weigh hazards of playing 'majon^po>itica■l role. 
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